Development of a probabilistic perception system for camera-lidar sensor fusion

La estimación de profundidad usando diferentes sensores es uno de los desafíos clave para dotar a las máquinas autónomas de sólidas capacidades de percepción robótica. Ha habido un avance sobresaliente en el desarrollo de técnicas de estimación de profundidad unimodales basadas en cámaras monoculares, debido a su alta resolución o sensores LiDAR, debido a los datos geométricos precisos que proporcionan. Sin embargo, cada uno de ellos presenta inconvenientes inherentes, como la alta sensibilidad a los cambios en las condiciones de iluminación en el caso delas cámaras y la resolución limitada de los sensores LiDAR. La fusión de sensores se puede utilizar para combinar los méritos y compensar las desventajas de estos dos tipos de sensores. Sin embargo, los métodos de fusión actuales funcionan a un alto nivel. Procesan los flujos de datos de los sensores de forma independiente y combinan las estimaciones de alto nivel obtenidas para cada sensor. En este proyecto, abordamos el problema en un nivel bajo, fusionando los flujos de sensores sin procesar, obteniendo así estimaciones de profundidad que son densas y precisas, y pueden usarse como una fuente de datos multimodal unificada para problemas de estimación de nivel superior. Este trabajo propone un modelo de campo aleatorio condicional (CRF) con múltiples potenciales de geometría y apariencia que representa a la perfección el problema de estimar mapas de profundidad densos a partir de datos de cámara y LiDAR. El modelo se puede optimizar de manera eficiente utilizando el algoritmo Conjúgate Gradient Squared (CGS). El método propuesto se evalúa y compara utilizando el conjunto de datos proporcionado por KITTI Datset. Adicionalmente, se evalúa cualitativamente el modelo, usando datos adquiridos por el autor de esté trabajoMulti-modal depth estimation is one of the key challenges for endowing autonomous machines with robust robotic perception capabilities. There has been an outstanding advance in the development of uni-modal depth estimation techniques based on either monocular cameras, because of their rich resolution or LiDAR sensors due to the precise geometric data they provide. However, each of them suffers from some inherent drawbacks like high sensitivity to changes in illumination conditions in the case of cameras and limited resolution for the LiDARs. Sensor fusion can be used to combine the merits and compensate the downsides of these two kinds of sensors. Nevertheless, current fusion methods work at a high level. They processes sensor data streams independently and combine the high level estimates obtained for each sensor. In this thesis, I tackle the problem at a low level, fusing the raw sensor streams, thus obtaining depth estimates which are both dense and precise, and can be used as a unified multi-modal data source for higher level estimation problems. This work proposes a Conditional Random Field (CRF) model with multiple geometry and appearance potentials that seamlessly represents the problem of estimating dense depth maps from camera and LiDAR data. The model can be optimized efficiently using the Conjugate Gradient Squared (CGS) algorithm. The proposed method was evaluated and compared with the state-of-the-art using the commonly used KITTI benchmark dataset. In addition, the model is qualitatively evaluated using data acquired by the author of this work.MaestríaMagíster en Ingeniería de Desarrollo de Producto

Development of a probabilistic perception system for camera-lidar sensor fusion

La estimación de profundidad usando diferentes sensores es uno de los desafíos clave para dotar a las máquinas autónomas de sólidas capacidades de percepción robótica. Ha habido un avance sobresaliente en el desarrollo de técnicas de estimación de profundidad unimodales basadas en cámaras monoculares, debido a su alta resolución o sensores LiDAR, debido a los datos geométricos precisos que proporcionan. Sin embargo, cada uno de ellos presenta inconvenientes inherentes, como la alta sensibilidad a los cambios en las condiciones de iluminación en el caso delas cámaras y la resolución limitada de los sensores LiDAR. La fusión de sensores se puede utilizar para combinar los méritos y compensar las desventajas de estos dos tipos de sensores. Sin embargo, los métodos de fusión actuales funcionan a un alto nivel. Procesan los flujos de datos de los sensores de forma independiente y combinan las estimaciones de alto nivel obtenidas para cada sensor. En este proyecto, abordamos el problema en un nivel bajo, fusionando los flujos de sensores sin procesar, obteniendo así estimaciones de profundidad que son densas y precisas, y pueden usarse como una fuente de datos multimodal unificada para problemas de estimación de nivel superior. Este trabajo propone un modelo de campo aleatorio condicional (CRF) con múltiples potenciales de geometría y apariencia que representa a la perfección el problema de estimar mapas de profundidad densos a partir de datos de cámara y LiDAR. El modelo se puede optimizar de manera eficiente utilizando el algoritmo Conjúgate Gradient Squared (CGS). El método propuesto se evalúa y compara utilizando el conjunto de datos proporcionado por KITTI Datset. Adicionalmente, se evalúa cualitativamente el modelo, usando datos adquiridos por el autor de esté trabajoMulti-modal depth estimation is one of the key challenges for endowing autonomous machines with robust robotic perception capabilities. There has been an outstanding advance in the development of uni-modal depth estimation techniques based on either monocular cameras, because of their rich resolution or LiDAR sensors due to the precise geometric data they provide. However, each of them suffers from some inherent drawbacks like high sensitivity to changes in illumination conditions in the case of cameras and limited resolution for the LiDARs. Sensor fusion can be used to combine the merits and compensate the downsides of these two kinds of sensors. Nevertheless, current fusion methods work at a high level. They processes sensor data streams independently and combine the high level estimates obtained for each sensor. In this thesis, I tackle the problem at a low level, fusing the raw sensor streams, thus obtaining depth estimates which are both dense and precise, and can be used as a unified multi-modal data source for higher level estimation problems. This work proposes a Conditional Random Field (CRF) model with multiple geometry and appearance potentials that seamlessly represents the problem of estimating dense depth maps from camera and LiDAR data. The model can be optimized efficiently using the Conjugate Gradient Squared (CGS) algorithm. The proposed method was evaluated and compared with the state-of-the-art using the commonly used KITTI benchmark dataset. In addition, the model is qualitatively evaluated using data acquired by the author of this work.MaestríaMagíster en Ingeniería de Desarrollo de Producto

Quantification of operating reserves with high penetration of wind power considering extreme values

The high integration of wind energy in power systems requires operating reserves to ensure the reliability and security in the operation. The intermittency and volatility in wind power sets a challenge for day-ahead dispatching in order to schedule generation resources. Therefore,the quantification of operating reserves is addressed in this paper using extreme values through Monte-Carlo simulations. The uncertainty inwind power forecasting is captured by a generalized extreme value distribution to generate scenarios. The day-ahead dispatching model is formulated asa mixed-integer linear quadratic problem including ramping constraints. This approach is tested in the IEEE-118 bus test system including integration of wind power in the system. The results represent the range of values for operating reserves in day-ahead dispatchin

Probabilistic Perception System for Object Classification Based on Camera -LiDAR Sensor Fusion

International audienceOne of the most basic needs to guide the definition of urban, agro-industrial and territorial management policies is to have a digital topographic representation or map of cities, crops and forests. These maps should ideally be created from multiple sensors whose responses are complementary (color information, for example, complements the returns of a LiDAR sensor in the presence of rain or low reflective objects). Once a topographic representation has been constructed, it can be used to produce and geo-localize higher-level estimates (e.g., location and classification of different trees and plants, crop density, location, and types of pests). Data can be collected using terrestrial unmanned vehicles equipped with hyper-spectral cameras, stereo cameras and LiDAR (Light Detection and Ranging) sensors. The processing of the acquired data can be used to generate a digital forest model (DFM). DFM will support forest planners in making multi-criteria decisions (MCDA) when planning harvesting operations. However creating a DFM or the map of a city, require a highly accurate and dense point cloud of the environment at hand. Motivated for building 3D reconstructions from which representations of different vegetation features of an environment can be obtained with high quality and precision. A robust perception system is proposed for densely predicting depth, since it is an essential component in understanding the 3D geometry of a scene. It is known that cameras provide near instantaneous capture of the workspace’s appearance such as texture and color, but from a single view, little geometrical information. On the other hand, laser readings may be so sparse that significant information about the surface is missing. The considerations above motivate the formulation of this work’s research question: How to develop a perception system for fusing a laser scan with a RGB image in order to produce a higher-resolution range

Network topological notions for power systems security assessment

The identification of topological vulnerabilities is a prerequisite for the study of security analysis. This paper presents a graph-theoretic framework to detect the minimum set of transmission lines interconnecting subnetworks inside of a power network. Moreover, the framework is used to develop a method to classify the criticality of substations. The approach can be used with power transfer distribution factors information to gain an insight about the power system security. Sometimes the power network exhibits high vulnerability related to critical transmission lines interconnecting critical substations from a physical point of view. The quantification of structural properties can provide meaningful information needed to assess and enhance the reliability and security of power system networks. The capabilities for the topological approach are illustrated on two large-scale networks. The proposed approach provides an effective tool for both real-time and offline environments for security analysis and contro